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Abstract 

We study the behavior of AS AT, a heuristic for solving satisfiability problems by stochas- 
tic local search near the SAT/UNSAT transition. The heuristic is focused, i.e. only variables 
in unsatisfied clauses are updated in each step, and is significantly simpler, while similar 
to, walksat or Focused Metropolis Search. We show that ASAT solves instances as large as 
N = 10 6 in linear time, on average, up to a = 4.21 for random 3SAT. For K higher than 3, 
ASAT appears to solve instances at the Montanari-Ricci-Tersenghi-Parisi "FRSB threshold" 
a s (K) in linear time, up to K = 7. 

1 Introduction 

Satisfiability problems (SAT) appear critically in many disciplines. Finding fast and reliable 
numerical methods for solving them is crucial in industrial applications, such as scheduling, or 
in verification. 

The random KSAT model, where each condition or clause has the same arity K, and an 
instance is picked randomly, has been of interest both to theoretical computer science and to 
statistical physics. For K > 3 KSAT belongs to the NP complete class of problems. While a 
deterministic algorithm will always find a solution if there is one, it presumably takes a long 
time to solve a KSAT instance in worst case [Q. However, the typical behavior can be different. 
Indeed, since the beginning of the 90s it has been known that the average running time of a 
deterministic algorithm depends on a = | the number of clauses (M) per variable (N) [2]. 
As a varies, a SAT/UNSAT transition is observed at a critical value a cr : below the threshold, 
a typical instance is satisfiable, while above a cr it is typically unsatisfiable. The transition 
becomes sharper as N increases [3]. Deterministic algorithms have (empirically) longest average 
run times close to a cr [21 E] . 

Stochastic search heuristics are not guaranteed to find a solution, if there is one, but may 
on the other hand greatly outperform a deterministic algorithm on a typical (solvable) instance. 
As a increases for given K and N, the typical run-time of a given heuristic increases, eventually 
diverging, at the latest at a cr . The most interesting behavior, if it can be established, is if for 



some heuristic the typical run-time grows only linearly in N for sufficiently small a, and also if 
the distribution of run-times per variable gets more narrow as iV increases. If so, run-time per 
variable is a self-averaging quantity. We denote here the greatest such a for some heuristic an n , 
the linear-time transition for that heuristic. 

A benchmark stochastic search is Papadimitriou's RandomWalksat [3]: in every step an 
unsatisfied clause is picked randomly, and then one random variable in that clause is flipped. For 
that algorithm, rate equations and direct simulations indicate that otn n on 3SAT is approximately 
2.7 [E1EJ- Furthermore, for simple heuristics, such as straight-forward guessing without back- 
tracking, rate equations and direct simulations also show a nonzero an n , albeit smaller [S|. 

A limitation of RandomWalksat is that it does not distinguish between which variable in 
a clause to flip, or if flipping one increases or decreases the number of unsatisfied clauses. 
The walksat [2] algorithm mixes RandomWalksat moves with greedy steps, by default in equal 
proportion. Walksat has been known to be quite powerful on SAT problems, but it was only 
shown quite recently to have a an n of 4.15 on 3SAT )lUj . In contrast to RandomWalksat, rate 
equations have not been set up for walksat: the interleaving of random and greedy moves, and 
the additional "freebie" move in the Selman-Kautz-Cohen heuristic, makes that complicated. 
Alava, Orponen and Seitz showed that au n for walksat could be pushed to or beyond 4.20 by 
optimizing over the proportion of random and greedy moves Furthermore, these authors 
showed that two other algorithms, Focused Metropolis Search (FMS) and Focused Record-to- 
Record Travel can be optimized to also have an apparent an n around 4.20. FMS in particular 
is quite simple: a variable in an unsatisfied clause is flipped if that decreases the number of 
unsatisfied clauses, and otherwise flipped or not flipped by a probability exponential in that 
number. FMS does not have the freebie move of walksat, but is still of comparable efficiency. 

In this contribution we will introduce and study a heuristic AS AT, for average SAT, which 
is arguably yet simpler than FMS. In ASAT a variable is flipped if this decreases the number 
of unsatisfied clauses, as in FMS, and then flipped with a constant probability if the number of 
unsatisfied clauses increases. ASAT is therefore sensitive to the widths of local minima, but not 
directly to the heights of the walls around local minima. 

The relevance of these studies lies along the following lines. First, more powerful search 
heuristics have a practical interest. While we do no not present detailed comparisons in this 
paper, let us state that ASAT generally runs somewhat faster than optimized FMS, which in 
turn is somewhat faster than optimized walksat with the Selman-Kautz-Cohen heuristic. More 
importantly, since ASAT is simpler than FMS, which in turn is simpler than walksat, one might 
hope for a analytical treatment along the lines of [HIE]. Second, from the theoretical side, it is 
of interest if au n for ASAT and other heuristics lie beyond 4.20 on random 3SAT, since that 
lies beyond two natural candidates for upper bounds on au n , known respectively as (which 
is around 3.92 for random 3SAT) and a s (around 4.15). 

The theoretical background of these two numbers and a s can be briefly described as 
follows. Within the cavity method, it has been shown that for low enough a the set of solutions 
is connected, while in the interval [a,i) a cr] the set breaks up into "clusters" |12l I13j . This was 
recently rigorously confirmed for large enough K |14| I15j . A satisfiability problem is equivalent 
to the problem of finding a zero-energy ground-state in a statistical mechanics model, where 
the "energy" is the number of unsatisfied clauses. In the UNSAT regime, where an instance is 
typically unsatisfiable, the ground-state energy is typically larger than zero. In the SAT regime, 
clusters of solutions are local minima, which are also global minima. If these are accompanied 
by a much larger number of clusters with non-zero energy, such clusters could act as traps to 
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local search heuristics. The number of clusters of local minima at given energy was computed by 
the cavity method for random 3S AT in j!2l I13| , and for higher K in , and does increase with 
energy. One possible conjecture, already falsified in 10 , would hence be that is an upper 
bound on au n . For KSAT a further phase transition takes place at a s , inside the interval [ad, a cr ], 
giving rise to a hierarchical structure of clusters |17| . For random 3SAT, a s is approximatively 
4.15. 

It is not clear to the present authors what the relation, if any, should be between a s and 
au n . Given however that au n for default walksat is larger than ad and very close to a s , one 
might conjecture that a s be an upper bound on a[ in . The results of this paper and of on 
the other hand indicate that au n for optimized algorithms is substantially larger than a s for 
3SAT. We show that run-time of ASAT is self-averaging at a = 4.21 on 3SAT up to instances of 
one million variables, while it is not self-averaging at a = 4.25. It is difficult to pin-point more 
precisely the transition point in this interval. It is even more difficult to compute the transition 
line of KSAT at K larger than 3 accurately, as the memory requirements per variable grow as 
Ka. We present here data that at a s (K), as computed recently in [Ij^, run-times of ASAT 
seem self- averaging up to K = 7. The time course of a solution is another quantity interest. 
Below aii n , ASAT solves an instance in linear time, similarly to Random Walksat below its au n . 
Above an n , but of course on a satisfiable instance, i.e. below a cr , ASAT typically solves an 
instance by a slow process, "sinking" through several plateaus. We show results from one such 
run, and we note that it appears to be different from the "solution by fluctuations" proposed 
for RandomWalksat above its au n 6 . To optimize ASAT we introduce a re-heating procedure, 
which also sheds light on the effective landscape seen by the algorithm. 

2 The ASAT heuristic 

ASAT is a focused heuristic, like RandomWalksat, walksat and FMS, meaning it focuses on the 
unsatisfied clauses at any given time, and makes a trial moves only to neighboring states by 
flipping a variable that appears in at least one unsatisfied clause. Variables that only appear 
in satisfied clauses are never flipped. In a sense, ASAT is perhaps the simplest extension of 
RandomWalksat. For any trial move, it only computes if that move will increase or decrease the 
energy (number of unsatisfied clauses). A move which increases the energy will be accepted with 
fixed probability, independent on how much the energy is changed, while a move that decrease 
the energy will be accepted always. In pseudo-code, ASAT is hence 

s = initial random configuration 

while t<t_max 

if F(s) = TRUE then EXIT 

at random pick a unsatisfied clause C 

at random pick a variable x in C 

x' = flip(x) 
s' = s(x -> X J ) 

if E(s') <= E(s) then flip x else 
flip x with probabililty p 
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Figure 1: Ranked logarithmic run-times per variable of ASAT, p = 0.21, at a = 4.21 and values 
of iV from 10 4 to 10 6 on 3SAT. Note pivoting of the distributions, as in |1U| . Note that all runs 
were made with a cut-off of 5 • 10 6 A r flips. Out of one hundred, all instances at N equal to 10 5 
and 10 6 are solved within this time, most instances at N = 10 6 taking close to 10 10 flips. For 
the smallest size, A' = 10 4 , the spread is larger, and about 10% of the instances are not solved 
in 5 • 10 10 flips, although the median is but a little more than 10 8 flips. 



The ASAT algorithm is therefore characterized by the single parameter p, which plays an analo- 
gous role to the proportion of random and greedy moves in walksat (a parameter also called p), 
and the noise parameter rj of FMS. Optimization of p in ASAT is discussed below in section 
Fig- n shows a rank ordered plot of the run-times for different system sizes A" = 10 , 10 5 , 10 6 at 
a = 4.21. This and analogous data for several lower values of a (data not shown) indicate that 
the run-time of ASAT is self-averaging at least this far. 



As an example of lack of self-averaging, we show in Fig. [2] a rank ordered plot at a = 4.25, the 
conjectured end point of the SID (Survey Induced Decimation) algorithm [131 112j . Clearly here 
there is no sign of self-averaging. 



The plot in Figgis in a sense misleading, as there are finite-size effects at smaller N. In other 
words, "good pivoting" and hence (numerically) convincing self- averaging is only displayed from 
values of A" around 10 4 and upwards. Similar finite-size effects were noted by the authors of 
on optimized FMS (data unpublished) and by one of us on walksat, and are presumably a feature 
of the constant-a ensemble. It seems that the onset of good pivoting increases with a, which 
makes a precise determination of au n difficult. The largest instances that fit into memory of 
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Figure 2: Ranked logarithmic run-times per variable of ASAT, p 
of N from 10 4 to 10 6 on 3SAT. 



0.21, at a = 4.25 and values 



present generation workstations is on the order of a million variables at a ~ a cr : there are aN 
clauses, each one of them specified by K integers, which for K = 3 and a ~ 4 makes about 50- A?" 
bytes. Hence, for a larger than 4.21 the interval between the onset of good pivoting and the 
largest instances that can be investigated is too small to draw a conclusion. For higher values 
K the problem quickly becomes worse, since a cr increases, roughly as 2 K . 

The solution process can be characterized by the fraction of unsatisfied clauses as a function 
of the number of flips. Following El IHj it is convenient to introduce a "time" as flips/N. 
Fig. El shows the time course of a solution process at a = 4.22. One can clearly see three 
regimes, one fast, one intermediate, and one quite slow. The fast regime, up to time about 
ten thousand, is presumably analogous to the Poissonian regime in Random Walksat as studied 
by EJ • The intermediate and slow regimes have, as far as we know, not been shown on 
this problem previously. We note that dynamics appears self-averaging in both the fast and 
the intermediate regime; perhaps hence both the fast and the intermediate regimes will be 
amenable to analysis. The slow regime proceeds by plateaus (long waiting periods). While this 
qualitative behavior repeats itself from run to run, the position and lengths of the plateaus do 
not. In the slow regime, dynamics is hence not self-averaging. We note that this solution mode, 
"slowly sinking in the energy landscape", seems qualitatively different from the "solution by 
fluctuations" found for Random Walksat in (El El- 



Finally, we have investigated ASAT for K larger than 3, albeit in less detail. While com- 
putational determination of the threshold gets harder at higher values of K, one may look for 
evidence that some given a is comfortably below au n . In Fig. [1] we have looked at a s (K), which 
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Figure 3: Time course of solution of one instance by ASAT at a = 4.22 and N = 10 6 . Note that 
ASAT solves this instance, but only after 1.3 * 10 5 time steps, i.e. 1.3 * 10 11 flips. The main 
plot, in logarithmic coordinates, shows that the solution proceeds in three stages. First, there 
is decay on a time scale up to about 10 3 . This process slows down, and is overtaken another 
process which last up to time about 2* 10 4 . Finally, there is a very slow decrease to the solution. 
Top insert shows a blow-up of the second stage in linear coordinates. Bottom insert shows the 
final decrease, which proceeds by plateaus, where the energy is approximately constant. In this 
run three plateaus can be discerned, with, approximately, 200, 50 and 15 unsatisfied variables, 
respectively. 
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Figure 4: Median and quartile run times per variable at a = a s (K), values taken from 16 . 
Figure indicates that ASAT solves these instances with about equal computational cost per 
variable, for all K. The parameter p was found by procedure ASAT-HEAT (see main text) on 
each value of K separately. The values were 0.21, 0.118, 0.068, 0.045 and 0.032 for K from 3 to 
7, respectively. 

values were recently given as 4.15, 9.08, 17.8, 33.6 and 62.5 for K from 3 to 7, respectively |16j . 
The results are not entirely conclusive, but tend to support that ctu n is greater than a s . 



3 State space structure and parameter optimization 

In this section we describe a method to optimize the value of the noise parameter p. The method 
called simulated heating of ASAT, or ASAT-HEAT, is also, as we will see, a useful tool when 
investigating the local barriers of random K-SAT. 

The idea is that there is a trade-off to be made between an algorithm getting out of local 
minima, and efficiently exploring the bottom of a local minimum. Hence, the premise is that 
solutions are found at the bottom of some local minima, which are not otherwise distinguished. 
After completion of this work, we became aware of a related idea, "optimization at the ergodic 
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edge", has recently been considered by Boettcher and Frank ^Hl; and also, to optimize the 
Record-to- Record- Travel algorithm, by Jia, Moore and Selman 

In the context of ASAT we look for the value p such that the algorithm does not get stuck, 
while still exploring the bottom of mimina where it finds itself. That is done by an interleaved 
process, where the algorithm alternatively runs with some non-zero p (to explore phase space, 
and get out of minima) , and alternatively freezes at zero p (to find the bottom of the mininum 
it is moving in). In ASAT-HEAT the value of p is raised incrementally in the following steps: 

• Generate an problem instance of size N close to a cr . 

• Run the heuristic with a low value of the parameter p and let the system equilibrate for 
some time r >> 1. We have used r equal to one thousand, i.e. 10 3 A flips. 

• Do a zero temperature quench, that is set the noise parameter to zero. The heuristic will 
then find a near local optimum by greedy search. 

• Reset the parameter p to its previous value, increase it by a small amount Ap, and let the 
system again equilibrate for a time r. 

• Iterate heating and quench. 

Fig.Elshows ASAT-HEAT for a system at K = 3, N = 10 4 and a = a c = 4.27. Up to p cr ~ 0.21 
the variations during the heating periods increase with p, while the lowest energies reached after 
the zero-temperature quench trend downward. Furthermore, the lowest energies seen during 
the heating periods are quite similar to the lowest energies after quench. This suggests that 
the algorithm is here in contact with the local structure, i.e. that the algorithm visits the local 
minima also during the heating periods. The downward trend of the lowest energies after quench 
indicates that the algorithm over time explores a larger set of minima, reaching lower energies, 
compare the time course of a simulation as in Fig. [21 

After p cr ~ 0.21, the variations during the heating periods are larger. The energies reached 
after quench are clearly lower than the ranges seen during the heating periods, indicating that 
the algorithm has now lost contact with the local structure. The energies reached after quench 
on the other hand increase with p, indicating that the algorithm now moves in higher-lying and 
more numerous local minima. We have found that ASAT works most efficiently using values of 
p at or slightly above p cr . 

To investigate how the algorithm moves in state space, we compute the Hamming distance 
between the state at some time (some value of p in ASAT-HEAT), and a reference state at the 
initial value of p. Fig. El shows the variation in Hamming distance for system size N = 10 4 , 
which exhibits a step- wise increase by plateaus. Fluctuations in Hamming distance are small 
below p cr , and increase rapidly above p cr . 

We have further investigated how p cr varies with N by running simulations at ten times the 
original system size. The overall behavior is similar and the transition seems to take place at or 
close to the same value of p. This suggests that the local small scale structure does not depend 
or depends only weakly on N. The value of p cr found by ASAT-HEAT decreases with K; the 
values for K from 3 to 7 are given in caption to Fig. 0] 
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Figure 5: Escape from meta-stable state at a = 4.27 and iV = 10 4 while increasing parameter p 
in procedure ASAT-HEAT (see main text). Note that up to a critical value p cr fluctuations are 
small and the energy decreases slowly. At higher values of p, fluctuations are larger, and energy 
increases with p. Figure shows every Nth time point during the heating periods (p greater than 
zero) and the zero-temperature quenches, at each value of p, respectively. 
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Figure 6: Same run as in Fig. El Hamming distance from initial meta-stable state increases with 
p by plateaus up to the critical value, at which is remains stationary. Fluctuations are relatively 
larger above the critical p. 

4 Discussion 

We have in this work presented a new heuristic for satisfiability problems, called AS AT. The 
heuristic has been implemented as an additional option in walksat (walksat distribution for- 
mat pOj, and is available upon request. One main interest of ASAT is that it is competitive 
with walksat and Focused Metropolis Search (FMS) on random KSAT problems close to the 
SAT/UNSAT transition, while being simpler, and hence (hopefully) amenable to analytic inves- 
tigations. We hope to return to this question in a future contribution. 

We have shown that ASAT has typical runtime linear in N up to a = 4.21 on 3SAT, up to the 
largest instances that can be studied (N ~ 10 6 ). We can not exclude that ASAT will eventually 
not look linear on even higher values of N, but we see no sign of such divergence. Finite-size 
effects make it difficult to show if or if not ASAT is linear or not beyond a = 4.21. This means, 
that to the best of our estimate, the linearity threshold for ASAT, an n , is for random, 3SAT 
larger than both the clustering transition (cy = 3.92) and the "FRSB threshold" (a s = 4.15). 
We have studied ASAT at larger K, and showed that probably au n is likely to be larger than 
a s there also. 

A parameter optimization technique, ASAT-HEAT, has been introduced. This allows for 
a determination of an optimal parameter value of the algorithm, and can be considered an 
alternative to the extensive simulations at many values of N, a and one algorithm parameter 
used in [TT] . 

While physical intuition suggests that local heuristics will have difficulties where many meta- 
stable states appear (at cy, or perhaps, more properly, at a s ) this does not seem to be the case. 
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RandomWalksat and very simple heuristics have difficulties far below ad, while AS AT and other 
heuristics seem to work linearly beyond a s . Let us therefore end by stating the differences 
between stochastic local search heuristics to find satisfying assignments in random KSAT and a 
physical process of random walk in a corresponding energy landscape. First, RandomWalksat, 
walksat, FMS and ASAT are focused: these algorithms correspond to non-equilibrium dynamics 
without detailed balance . Conservation of any cluster structure at all under non-equilibrium 
perturbations has apparently been a moot point in some spin glass models |21U22| . Second, while 
FMS is similar to a random walk in the energy landscape, in the sense that the dynamics directly 
depends on the local energy, walksat and ASAT and obviously RandomWalksat do not. In 
walksat with Selman-Kautz-Cohen heuristic, decisions are based on the change in "breakclause" 
which is not the same as the energy change, while in ASAT decisions are based on whether 
the energy increases or decreases at all. Therefore, finally, while numerical simulations cannot 
rule out that e.g. ASAT will run into trouble beyond a s on instances larger than the ones we 
have studied, we are not sure if it necessarily has to. Theoretical predictions on what iV one 
would expect to see nonlinear behavior at what a would be most helpful to guide numerical 
experiments on this issue. 
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